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Summary 

The  research  described  below  was  carried  out  during  the  period  1 
February  1998  to  30  September  2001.  We  have  made  significant  progress 
on  several  fronts.  We  have  obtained  positive- weight  quadrature  rules  that 
are  exact  for  spherical  harmonics  of  prescrided  order  and  that  allow  func¬ 
tion  evaluations  at  scattered  points,  and  we  have  given  algorithms  for 
obtaining  these  weights.  Based  on  these  rules,  we  were  able  to  construct 
neural  networks  for  spheres  using  zonal  activation  functions.  We  also 
made  progress  on  the  difficult  problem  of  locating  multiple  sources  with 
neural  networks.  On  another  front,  we  provided  error  estimates  for  in¬ 
terpolating  less  smooth  functions  via  networks  with  smooth  activation 
functions;  this  is  the  first  result  of  its  kind.  In  addition,  we  provided  a 
class  of  functions  to  which  our  error  estimates  aplly;  these  functions  are 
both  easy  to  use  and  locally  supported,  so  that  interpolation  matrices 
arising  from  them  will  be  banded. 


1  Review  of  Research 

The  research  described  below  was  carried  out  during  the  period  1  February 
1998  to  30  September  2001,  the  final  year  being  a  no-cost  extension.  The  work 
itself  is  communicated  in  the  papers  and  informal  technical  reports  listed  in  §2. 
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Publications  are  denoted  by  ‘P’,  technical  reports  by  ‘T’.  The  numbering  is  from 
the  lists  in  §2.1  and  §2.2.  Our  research  fits  into  the  broad  categories  described 
below. 

1.1  Error  estimates  and  positive- weight  quadrature 

Our  research  on  error  estimates  for  generalized  Her  mite  interpolation  using 
radial-like  basis  functions  on  non-traditional  spaces  -  the  n-sphere,  n-torus, 
and  Riemannian  manifolds  -  began  with  work  done  jointly  with  N.  Dyn  that  is 
described  in  [P2].  These  rates  were  the  first  for  such  manifolds,  and  made  use 
of  variational  techniques  in  a  reproducing  kernel  Hilbert  space  setting.  For  the 
sphere,  these  results  depended  on  a  sampling  theorem  for  the  sphere  introduced 
by  Driscoll  and  Healy  [2].  The  use  of  this  sampling  theorem  limited  the  applica¬ 
bility  of  these  results  to  the  latitude-longitude  grid  on  the  sphere.  At  the  time, 
one  of  our  goals  was  to  expand  the  range  of  applicability  of  this  method.  How 
we  did  this  is  described  below. 

We  also  completed  joint  work  with  R.  Schaback  [P12]  investigating  a  version 
of  a  multilevel  method  introduced  by  Floater  and  Iske  [1]  that  interpolates 
residuals,  but  uses  convolutions  of  RBFs  rather  than  the  scaled  RBFs  employed 
in  [1].  The  methods  employed  were  based  on  ones  used  in  [P2].  We  obtained 
rates  of  approximation  for  this  interpolation  process.  These  estimates  were 
the  first  ever  for  cases  in  which  parameters,  such  as  spreads  and  actual  basis 
functions,  were  allowed  to  change. 

Building  on  the  framework  established  by  Dyn,  Narcowich,  and  Ward  in  [P2], 
and  joint  with  Professors  Jetter  and  Stoeckler,  we  obtained  rates  of  approxima¬ 
tion  by  spherical-basis-function  (SBF)  interpolants  for  the  case  of  scattered- data 
on  the  d-sphere  [P3].  The  work  introduced  the  Banach-space  idea  of  norming 
sets  to  surface  fitting  problems.  The  bounds  obtained  were  explicit,  in  terms 
of  the  mesh  norm  of  the  data.  In  addition,  new  quadrature  formulas,  based  on 
scattered-data,  were  provided  for  the  sphere  in  [P4],  but,  unfortunately,  these 
gave  no  control  on  the  weights  in  the  quadrature  formulas. 

This  lead  to  a  deeper  investigation  of  the  possibility  of  obtaining  positive 
weight,  scattered-data  quadrature  formulas  for  the  d-sphere.  These  formulas 
would  reproduce  spherical  harmonics  of  a  given  order,  but  at  the  same  time 
use  a  number  of  points  from  the  scatted  data  comparable  to  the  dimensions  of 
the  spaces  of  spherical  harmonics  involved.  Such  results,  which  are  well  known 
and  have  long  been  utilized  for  special  point  sets  in  intervals  on  the  line  and 
in  rectangular  boxes  in  higher  dimensions,  were  not  known  for  the  sphere.  In 
[P6],  which  was  joint  with  H.  Mhaskar,  we  not  only  obtained  such  formulas,  but 
we  arrived  at  them  by  establishing  an  important  norm  equivalence.  We  showed 
that,  for  the  space  of  spherical  harmonics  on  having  order  n  or  less,  the 
LP{S'^)  norm  is  equivalent  to  the  discrete  P  norm  of  the  spherical  harmonics 
restricted  to  a  point  set  X  e  where  the  number  of  points  in  X  is  compa¬ 
rable  to  the  dimension  of  the  space  of  spherical  harmonics.  This  generalizes  to 
the  sphere  an  old  result  of  Marcinkiewicz  and  Zygmund  for  trigonometric  poly¬ 
nomials.  One  spinoff  of  the  norming-set  technique  is  a  scattered-data  positive 


2 


weight  quadrature  formula  on  a  cube  in  and  related  approximation  results; 
these  were  reported  in  [P8].  Another  concerns  neural  networks,  which  we  will 
describe  in  §1.2  below. 

Others  applications  of  norming  sets  again  deal  with  error  estimates.  RBF 
error  estimates  on  compact  domains  iii  were  first  obtained  for  thin-plate 
splines  by  Duchon  [3,  4]  and  for  RBFs  in  general  by  Madych  and  Nelson  [5,  6]. 
Recently,  Wendland  [14]  pointed  out  that  the  unevaluated  constants  in  these 
papers  can  be  estimated  using  norming  sets,  and  did  so  for  the  Gaussian  RBF. 
In  joint  work  with  Wendand  [T2]  we  employed  the  norming  set  technique  to 
provide  new  and  refined  error  estimates  that  apply  to  scattered-data  interpolants 
and  their  derivatives,  not  only  in  R^bn  t  on  the  d-torus  and  2-sphere  as  well. 
In  many  cases  our  estimates  provide  bounds  orders  of  magnitude  smaller  than 
those  previously  known. 

Many  of  the  results  mentioned  above  dealt  with  error  estimates  occurring 
when  the  functions  sampled  belong  to  the  reproducing  kernel  Hilbert  space 
induced  by  an  RBF  or  similar  kernel.  These  spaces  are  often  called  native 
spaces.  The  overall  effect  is  that  these  results  are  limited  to  having  the  functions 
sampled  be  as  a  smooth  as  the  RBF,  itself.  In  many  cases  this  is  not  a  problem. 
However,  when  the  smoothness  class  of  a  target  function  is  unknown,  this  does 
become  a  problem. 

In  work  done  jointly  with  R.  Schaback,  we  began  addressing  this  problem. 
We  showed  in  [P13]  that  we  could  approximate  target  functions  outside  of  the 
native  space  using  linear  combinations  of  RBFs.  The  linear  process  we  provided 
in  that  paper,  while  yielding  good  results,  is  difficult  to  implement.  We  felt  we 
could  do  better. 

The  key  to  doing  better  is  a  technique  “dual”  to  the  one  employing  norming 
sets;  it  was  presented  in  [PlOj.  This  technique,  which  is  functional  analytic  in 
character,  allows  one  to  obtain  simultaneous  interpolation  and  a  near-optimal 
degree  of  approximation  by  radial  and  related  basis  functions,  for  a  cube,  torus 
or  2-sphere.  Related  results  are  also  found  in  [T3]. 

Our  most  recent  work  [T5]  applies  the  “dual”  norming  set  technique  to  ob¬ 
tain  Sobolev-type  error  estimates  for  interpolating  functions  /  e  from 

“shifts”  of  a  smoother  SBF  defined  on  Moreover,  these  estimates  are  close 
to  the  optimal  approximation  order,  and  obtaining  them  is  computationally  fea¬ 
sible.  Although  RBF  interpolation  has  been  in  in  use  for  nearly  a  score  of  years, 
no  previous  work  has  successfully  shown  that  RBF-type  interpolants  converge 
when  the  sampled  function  is  not  in  the  native  space.  This  is  the  first  paper  to 
do  so. 

1.2  Neural  networks  and  PDEs 

The  field  of  neural  networks  encompasses  a  vast  area,  overlaps  with  many  oth¬ 
ers  fields,  and  can  be  approached  in  a  variety  of  ways.  Our  approach  is  that 
of  Poggio  and  Girosi  [9],  who  view  learning  as  a  problem  of  hypersurface  re- 
construction.  In  a  broad  sense,  all  of  the  work  we  have  done  has  been  aimed 
at  this  problem.  Even  so,  we  wish  to  discuss  results  that  specifically  apply  to 
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neural  networks,  especially  ones  on  non-traditional  spaces  —  spheres  and  other 
manifolds. 

Neural  networks  on  spheres  come  up  in  applications  to  geophysical  prob¬ 
lems.  Networks  on  other  spaces  that  we’ve  studied  arise  in  connection  with 
neural  beamforming  problems  [8,  11].  Our  paper  [P15],  joint  with  O’Donnell 
and  Southall,  details  our  work  in  attacking  the  problem  of  direction  finding  in 
the  presence  of  multiple  sources  and  sets  out  new  avenues  for  future  research. 
Concerning  the  sphere,  we  have  written  several  papers  addressing  questions  con¬ 
cerning  stability,  interpolation,  and  approximation  with  networks  using  zonal 
functions  as  activation  functions  for  the  network.  In  [P14],  we  obtain  stability 
estimates-i.e.,  norms  of  inverses  of  interpolation  matrices  and  condition  numbers 
for  these  matrices.  More  recently,  we  employed  the  quadrature  formulas  from 
[P6]  to  study  approximation  power  for  zonal  function  networks;  these  results, 
which  are  reported  in  [P5],  showed  that  the  networks  themselves  have  nearly 
optimal  Sobolev- type  approximation  properties.  In  [P9]  and  [T4],  we  construct 
a  multiresolution  analysis  of  the  standard  Hilbert  space  on  a  Euclidean  sphere, 
which  can  be  implemented  directly  by  neural  networks.  The  neural  networks 
may  utilize  any  sufficiently  smooth  function  as  an  activation  function,  and  their 
size  can  be  determined  in  advance.  We  introduce  frame  operators  that  can 
analyze  data  selected  at  scattered  sites.  These  frames  can  be  used  to  detect 
singularities,  even  in  higher  order  derivatives. 

Papers  dealing  with  applications  of  RBFs  to  solving  PDEs  numerically  have 
for  the  most  part  been  experimental.  Definite  theoretical  results  on  error  es¬ 
timates  and  stability  have  been  looked  at  in  very  few  cases,  and  then  only  for 
elliptic  problems,  which  can  be  solved  via  finite  element  methods.  In  joint  work 
with  R.  Lorentz  [Tl],  we  present  RBF/Hermite  collocation  methods  that  are 
adaptive,  highly  flexible,  multivariate,  and  can  be  employed  in  non-traditional 
spaces,  the  sphere  for  example.  We  analyze  in  detail  a  simple  transport  problem 
solved  via  such  a  method,  and  we  found  that  it  provides  spectral  convergence 
orders  that  are  competitive  with  previously  known  polynomial  methods,  which 
have  very  limited  scope  and  are  not  adaptive.  This  raises  important  questions 
for  future  work. 

1.3  Locally  supported  basis  functions 

To  describe  the  results  that  we  have  gotten,  we  need  to  provide  some  background 
material.  A  positive  definite  function  on  a  sphere  is  a  function  </>(cos^),  where 
0  is  the  geodesic  (great  circle)  distance  between  two  points  on  a  sphere.  The 
idea  is  that  the  interpolation  matrices  with  entries  0(cos0j,fc)  are  positive  semi- 
definite  for  an  arbitrary  finite  set  of  scattered  points  on  the  sphere.  Long  ago 
Schoenberg  [10]  characterized  such  functions  as  being  those  having  Legendre 
expansions, 

oo 

^(cos^)  =  (cos  6>), 

^=0 
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in  which  a£  >  0  and  Pi{cos  0)  is  the  degree  Legendre  polynomial  correspond¬ 
ing  to  the  n-sphere.  When  n  =  2,  these  are  the  standard,  familiar  Legendre 
polynomials.  When  n  7^  2,  they  are  proportional  to  Gegenbauer  polynomials 
[7].  To  guarantee  that  the  interpolation  matrices  associated  with  0  are  positive 
definite  and  thus  are  invertible,  we  impose  the  condition  that  a£  >  0  for  all  £, 
A  (j)  satisfying  this  is  a  spherical  basis  function  (SBF). 

There  are  three  ways  of  obtaining  SBFs:  (1)  directly  from  the  series;  (2)  from 
functions  having  known  expansions,  such  as  ^  or  a  generating  function;  and, 
finally,  (3)  from  restricting  RBFs  on  Euclidean  spaces  to  a  sphere.  The  difficulty 
with  (1)  is  that  either  one  has  to  sum  an  infinite  series  or  obtain  good  numerical 
approximations  to  its  sum.  Most  of  the  SBFs  that  we  have  came  by  way  of  (2), 
Somewhere  along  the  line  a  series  expansion  for  a  function  was  known,  and  the 
coefficients  were  observed  to  be  strictly  positive. 

Restricting  RBFs  to  the  sphere  is  simple  enough.  Suppose  that  x^y  £ 
and  that  the  angle  between  x  and  y  is  0,  so  that  the  Euclidean  dot  product 
a;  ■  y  =  cos  6^.  If  we  let  ^  be  an  RBF  in  then 


0(x  -y)  :=  ^(||a;-y||2)|a:,ye5- 

is  easily  seen  to  be  positive  definite  on  S^.  The  hard  problem  was  determin¬ 
ing  whether  or  not  whether  or  not  a  </>  gotten  in  this  way  was  an  SBF-that 
is,  whether  or  not  all  its  Legendre  coefficients  were  strictly  positive.  In  [T5], 
we  showed  that  under  very  mild  conditions,  such  (j)  were  SBFs.  This  opened 
the  door  to  introducing  a  new  class  of  SBFs;  these  are  locally  supported  func¬ 
tions  arising  via  restrictions  of  Wendland’s  compactly  supported  RBFs  [12,  13]. 
These  new  functions  are  quite  attractive;  they  can  be  both  explicitly  and  easily 
computed  and  are  also  have  good  convergence  properties.  Future  work  would 
involve  further  exploration  of  their  properties  and  fast  evaluation  methods. 
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1.  R.  Lorentz,  F.  J.  Narcowich,  and  J,  D,  Ward,  Collocation  Discretizations 
of  the  Transport  Equation  with  Radial  Basis  Functions,  preprint. 

2.  F.  J.  Narcowich,  H.  Wendland  and  J.  D.  Ward,  Refined  Error  Estimates 
for  Radial  Basis  Function  Interpolation,  preprint. 

3.  F.  J.  Narcowich,  N.  Sivakumar  and  J.  D.  Ward,  On  Convergent  Interpo- 
latory  Processes  Associated  with  Periodic  Basis  Functions,  preprint. 

4.  H.  N,  Mhaskar,  F.  J.  Narcowich,  and  J.  D.  Ward,  Neural  Network  Frames 
on  the  sphere,  preprint, 

5.  F,  J.  Narcowich  and  J.  D.  Ward,  Scattered-Data  Interpolation  on  Spheres: 
Error  Estimates  and  Locally  Supported  Basis  Functions,  preprint. 

2.3  Research  Conference  Talks 

F.  J.  Narcowich 

•  Narcowich  gave  a  plenary  address  at  the  9TH  INTERNATIONAL  CON¬ 
FERENCE  ON  APPROXIMATION  THEORY,  3-6  January  1998,  Van¬ 
derbilt  University,  Nashville,  TN  (C.  Chui  and  L.  Schumaker,  organizers) 

•  Narcowich  gave  an  invited  half-hour  talk,  “Remarks  on  scattered-data  sur¬ 
face  fitting  via  positive  definite  kernels,”  EILAT98  International  Confer¬ 
ence  on  Multivariate  Approximation  and  Interpolation  with  Applications 
in  CAGD,  Signal,  and  Image  Processing,  held  7-11  September  1998  in 
Eilat,  Israel 

•  Narcowich  gave  an  invited  half-hour  talk,  “Scattered  data  quadrature  for 
spheres,”  Session  on  Mathematical  Methods  of  Geodesy  at  the  Mathe- 
matiches  Forschungsinstitut  Oberwolfach,  29  March  -  3  April  1999.  The 
session  was  organized  by  W.  Freeden,  E.  Grafarend,  and  L.  Svensson. 

•  Narcowich  gave  an  invited  twenty-minute  talk,  “Multiple  source  direction 
finding  with  reduced  training  and  increased  generalization,”  at  the  Mil¬ 
lennium  Conference  on  Antennas  k.  Propagation,  held  from  9-14  April 
2000,  Davos,  Switzerland. 

•  Narcowich  gave  an  invited  twenty-minute  talk,  “Scattered  Data  Interpo¬ 
lation  on  Spheres:  Locally  Supported  Basis  Functions,” at  the  Special  Ses¬ 
sion  on  Sphere  Related  Approximation  and  Applications,  AMS  Regional 
Meeting  in  Chattanooga,  TN,  October  5-6,  2001. 
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J.  D.  Ward 

•  Ward  gave  an  invited,  one-hour  address  at  the  conference  on  CADG  AND 
WAVELETS  held  at  Montecatini,  Italy,  15-17  June  1998 

•  Ward  gave  an  invited  half-hour  talk,  “Remarks  on  qudrature  formulas 
for  the  n-sphere,”  EILAT98  International  Conference  on  Multivariate  Ap¬ 
proximation  and  Interpolation  with  Applications  in  CAGD,  Signal,  and 
Image  Processing,  held  7-11  September  1998  in  Eilat,  Israel 

•  Ward  gave  an  invited  one-hour  talk,  “Approximation  from  spaces  of  shifts 
of  a  positive  definite  kernel,”  International  conference  in  Approximation 
Theory,  held  28  Sept  -  2  Oct  in  Dortmund,  Germany 

•  Ward  gave  a  one-hour  invited  talk,  “Some  remarks  on  wavelets  on  the  m- 
sphere,”  given  at  the  International  Conference  on  Wavelet  Analysis  and  Its 
Applications,  held  from  15-20  November  1999  at  Zhongshan  University, 
Guangzhou,  P.  R.  China 

•  Ward  gave  a  contributed  talk,  “Convergent  interpolatory  processes  asso¬ 
ciated  with  periodic  basis  functions,”  at  the  symposium  “Trends  in  Ap¬ 
proximation  Theory”  held  from  17  -  20  May  2000  at  Vanderbilt  University, 
Nashville,  TN. 

•  Ward  gave  a  contributed  talk,  “Approximation  with  interpolatory  con¬ 
straints,”  at  the  TENTTH  INTERNATIONAL  CONFERENCE  ON  AP¬ 
PROXIMATION  THEORY,  26-29  March  2001,  University  of  Missouri  at 
St.  Louis,  St.  Louis,  MO  (C.  Chui  and  L.  Schumaker,  organizers). 

•  Ward  gave  an  invited  colloquim  on  Wednesday,  November  13  at  the  Uni¬ 
versity  of  Georgia,  Athens,  GA. 

•  Ward  gave  an  invited  twenty-minute  talk,  “Scattered  Data  Interpolation 
on  Spheres:  Approximating  Rough  functions  by  Smooth  Kernels,” at  the 
Special  Session  on  Sphere  Related  Approximation  and  Applications,  AMS 
Regional  Meeting  in  Chattanooga,  TN,  October  5-6,  2001. 

3  Activities 

•  Dr.  Ward  visited  Major  O’Donnell  at  Hanscomb  AFB  on  31  July  1998. 
He  discussed  a  number  of  mathematical  problems  concerning  location  of 
multiple  sources  using  neural  networks  and  artificial  arrays. 

•  Narcowich  visited  Terry  O’Donnell  at  Hanscom  from  17  May  to  19  May 
1999.  The  purpose  of  this  visit  was  to  discuss  various  aspects  of  detection 
of  multiple  sources  via  neural  beamforming.  The  details  of  this  visit  were 
given  in  a  report  to  Dr.  Nachman,  written  on  24  May. 


8 


•  Narcowich  and  Ward,  and  Robert  Schaback,  from  Goettingen  University 
in  Germany,  spent  two  weeks,  6  August  2000  -  19  August  2000,  at  the 
Mathematische  Forschungsinstitut  Oberwolfach,  under  the  auspices  of  the 
Volkswagen-Stiftung  “Research- in-Pairs”  program.  Professor  Robert  Sch¬ 
aback  is  a  leader  in  the  field  of  radial  basis  functions. 

•  Narcowich  and  Ward,  along  with  H.  Mhaskar,  were  asked  by  the  organiz¬ 
ers  of  the  EILAT98  International  Conference  mentioned  above  to  write  a 
survey  article  to  be  appear  in  a  book.  The  survey  article  that  resulted  is 
listed  as  [P7]. 

•  Ward  attended  the  CBMS  lecture  series  on  wavelets,  from  22  -  26  May 
at  the  University  of  Missouri  in  St.  Louis;  David  Donoho  was  principal 
speaker. 

•  Narcowich  attended  an  IMA  Workshop  on  Geometric  Design,  held  from 
23  April  to  30  April  2001  at  the  University  of  Minnesota,  and  organized 
by  L.  Schumaker. 
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